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® Motion dependant video signal processing. 



® In, for example, a motion compensated video standards converter wherein blocia in a first field or frame of a 
video signal are each compared wth a plurality of btocks in ttie next succeeding field or frame of the video 
signal for deriving motion vectors represenfing the motion of the content of respective blocks tietween the first 
field or frame and the next fieW or frame, the blocks (a; q) adjacent to a border region of the field or frame are 
assigned moHon vectors selected from the molton vectors which have been derived fbr those of the blocks (d, b, 
c: r. ^ 8. t. g) which are adiaoert to the block (a. q) iindar consideraB^ 

FIG. 34. 
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MOTION DQiENDEm-VIDeo 8M3NAL PflOCESSiNa 



This invention reiaies to mOOm depemdant video signal processing. More particularty. «ie invention 
relates to video standards convorters using such «ignai prorassing. and to meBwds of deriving motion 
vectors representing motion iwtween fields or frames of a video signal. 

Video standards converters are well Icnown devices used to convert video signals from one standard to 
5 another, for examfde, from a 92S Rnes per frame, 50 fields per second stendard to a 525 lines per frame, 60 
fields per second standard. Video standards ccmversion cannot l>e achieved satistactoriiy merely by using 
simple linear interpolation techniques, t)ecause of the femporaf and vertical alias whidi is present in a video 
si^al. Thus, simple finear Interpolation produces unwanted artotads in the resulting pi^ure, in particuiar, 
the pictures are blurred verticaliy and judder tsmporaiiy. 
TO To reduce these |)n)bieins it has bem proposed Hot video standards converters alKMild use adapfive 
techniques to switch tlw peramelnrs of a Inear interpolator in dependence on the degree of movement in 
the pteture represented by the incombig video signal. 

It has sdso been proposed, for example for the purpose of data reduction in video signal processing, to 
generate motion vectors from an Incoming video signal by a block matching technique, in which the content 
T6 of a seanch blocic in one Field or frame is compared with the respective contents of a plurality of search 
btotiks comprised in a search area in the following field or frame, to determine the minimum difference 
between the contents so compared, and hence the direction and distance of motion (if any) of the content 
Of the original search blocic 

The present invenlian is particularly concerned with the problem of providing ntotion vsct!>rs for search 
20 blocks which are aciyacent to a txxder rsgkxi of the field or frame. 

According to the present invention there is provided a motion compensated video standards converter 

means for comparing biocics in a first field or frame of a video Signal with a plurality of bloclcs in the 
foUowing field or frame of the video signal for deriving motion vectors r^iresenting the motion of the content 

as (rf respective said blodts between said first field or fraine and said fblloiw^ 

means to assign to each of said blocks atigacent to a border rag^ of each said fieM or frame, motion 
vectors selected from motion vectors whk:h have been darhied tor those of uid blocks which are adjacent 
to said titodk uncter cohskieratkm and are not in sak) border regkm; and an interpolator oontroOed in 
dependence on sakJ motkxi vectors. 

30 AccoRftig to llw present Imenfion there is also provkled a method of deriving moOon vectors 
representing motion between successive fieMs or frames of a vktoo signal, the method inchidins the ste^ 
of: 

comparing btocKs in a first field or frame of the vfcleo signal wHh a plurality of blocks in ttie folbwing field or 
frame of the video signal for deriving said molkm vectors representing the motion of the content of 
3S rsspectwe said bkicks between said first fiekl w frame and sak) following field or frame; and 

assigning to each of said btocks adjacent to a bonier regton of each said fiekl or frame, the motkm vectors 
selected from the motion vectors whteh have been derived for those oi saM blocks vyhich are adjacent to 
said block under consWeration and are not in sakl border region. 

The invention will now be described by way of example vrith reference to ttie accompanying drawings, 
40 throughout which like parts are refenad to by like references, and in whkdK 

Figure 1 is a block dtagram of an embodimeiit of motton comperisated video standards converter 
according to the present inventkm; 

Figure 2 shows diagrammaOcaliy progressive scan conversion; 

Rgures 3 to 6 show diagrammaHcally sequences of lines in sequences of fiekls for explaining 
45 progressive scan convertion; 

Figure 7 is a btock diagram showing the ste(» in motion adaptive progressive scan conversion; 

Figure 8 shows diagrammaltoally progrerahn scanning, in particular the required estimate and 
difference value between succesdve fieUs; 

Figures 9 and 10 are diagrams used hi explaining the technk|ue of Figure 8 in more detail, f^ure 9 
so showing a progress sew normaOzing ftncUon and Figure 10 showing a progressive scan non-lnear 

Figure 11 shows dSagrammatkally the creation of pixels in mtesing Rnes in progressive scan 
converskxi; 

Hgure 12 and 13 show diagrammalteally search btocks and search areas, and the relaltonships 
therebetween; 
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Figure 14 shows a carmalallon surface; 

Figures 15 and 16 show diagranunalicayy how a search blodc Is grown; 

Figure 17 shows the areas of a frame in which ssach blocic nudching is not possMe; 

Rgure 18 shows diagrammatically a moving object straddling three search blocks; 
6 Figures 19 to 21 show three resjiting correialion surfaces, respectively; 

Figures 22 and 23 show further escanples of conrelaiion surfaces, used in describing a ttve^ld test; 

Rgures 24 and % show ^11 further ocamples of conelation surfaces, used in descrfUng a rings test; 

Figure 26 shows diagramiDsticany bow the direction in which a search block is to grow is determined; 

Figure 27 shows dtagrammaOcally how a conrelafion surfoce is weighted; 
TO Figure 28 shows the relaSonship between sample blocfcs and search bloclcs. and a frame of video; 

Figure 29 shows motion vector regions in a frame of \^deo; 

Figures 30 to 32 show dl^ranra tsed in explaining mofion vector reduction in respective regions of a 
frame of video; 

Rgure 33 shows part of the embodiment in more detailed blocic diagrammatic form; 
rs Rgure 34 is a (£agram used in explaining motion vector js^nment to search bfode which are 
adjacent to a border region of a frame: 

Rgure 35 is a flow chart of the operation of the arrangement of Rgure 33; 

Rgures 36 and 37 show diagrammatically a first stage in motion vector selection; 

Figures 38 and 38 show dte«rammaflcally how a threshold is established during the motion vector 

Figure 40 shows diagrammatically a second stage in motion vector selection; 

Figures 41 to 47 show arrays of pixels with assodalad motion vectors, used in explaining motion 
vector post-processing; and 

Figure 48 shows diagrammatically the operation of an interpolaior. 
26 The embodiment of mofion compensated video standards converter to be described is particularly 
intended for use in the conversion of a high delMtian video signal (HDVS) having 1125 lines per frame. 60 
fields per second, to 24 frames per second 3Smm film. However. It wil be understood that the invention is 
not fimiied in this respect, and that the standards converter can readHy be adapted to offset conversions 
between other standards. 

30 Figure 1 Is a block diagram of the standards convener. The standards converter comprises an input 
terminal 1 to wftfadi an input video signal is suppGed The input ttmiinal is connectod to 
converter 2 in which the input video fields are converted Into video frames which are supplied to a direct 
btoci( matoher 3 wherein correlation surfaces are created. These coaelation surfaces are analysed by a 
motion vector estimator 4, which derives and supplies mofion vectors to a motion vector reducer 5, wherein 

35 the number of motion vectors for each pixel is reduced, before titey are suppliad to a motion vector selector 
6, which also receives an outout from ttte progressive scan converter 2. Any irregularity in tiie selection of 
ttie motion vectors by the motion vector selector 6 is removed by a motion vector post processor 7, from 
which the processed motion vectors are suppfied to and control an interpolator 8 which also receives an 
input from the progressh« scan converter 2. The output of the interpolator 8. which is a standards- 

40 converted and motion-compens a ted video signsi is supplied to an output terminal 9. Each part of the 
standards converter and the operation thereof will be described In more detail l>elow. 

The progressive scan converter 2 produces output frames at the same r^ as flie input fields. Thus, 
referr^ to Figure 2 which shows a seqMence of consecutive lines in a sequence of consecutive fields, the 
crosses representing lines present in the input fields and the squares representing interpolated lines, each 

4s output frame will contain twice the number of ines as an input field, the lines alternating between lines from 
the input video signal and Snes which have been Intorpolatod by one of tt» methods to be described below. 
The interpdatad lines can be regarded as an interpolated field <rf «ie opposite polarity to the input field, but 
in the swne temporal position. 

Progressive scan conversion is preterably carried out for two main reasons; firstiy. to maice tiis 

SO following direct tilocic malfihing process easier, and secondly in coraiderBtion of the final output video 
loimaL Tliese two reasons will now be considered in more detalL 

Direct blocic matehing is used to obtain an aocwato estimation of the horizontal and vert^l motion 
between two successive video fields, as described in mom detail below. However, due to ttie interlaced 
sbxicture of ttie video signal on which direct block mirtching is performed, probtems can arise. 

99 Consider the image represeined by F^uio 3. whteh indicates a sequence of successhre lines in a 
sequence of successive fleWs. the open squares reprssanttng wliite pbcets. the Uacfc squares rBpresenting 
Uaefc l^x^ and the Islched squares rspreserrting grey pbeels. TMs, therefore, represents a sbi& ptoture 
wtm a hlg^ vertical frequency component which in a HDVS would be 112SO cydes per pictwe height As 
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ttds image has been sampled by the usual intarlace scanning procedure, each field appears to contain a 
st^ vertical frequency luminance component Y of 112&6 cph. as indicated in figure 4. However, the 
frequency components in each field are seen to be in anti-phase. Attempts to perform direct block matching 
between these two fields mil lead to a number of diffisrent vaki^ for the vertical motion component, ail of 

5 which are inconect This is indicated in Figure 5, in which the abbreviation LPF means lines per field. From 
Figure 5 it is dear VnA direct block matching will not the cotxetit answer for the vertical motion 
component, wHch component should hi tact be zero. This is because the direct blodc matching is in fact 
trscking 4ie alias component of the video signal raOier than the actual motion. 

Consider now Figure 8, which depicts the same static ftni^ as Rgure 3. mcept that now each input 

10 field has been progressive scan converted to form a frame, the trfangtes representing interpolated pixels. It 
can tts seen ttiat each frame now contains the same static vertical frequency component as the original 
input fields, «)at is 11^ cph. Thus, direct blodc matdiing between two successive frames can now give 
the conrect value for the verticai nwtion, that is, zero, and the traddng of the vertical alim 
DAoreover, there is the point that direct block maldiing on progressive scan converted frames win result in a 

js more accurate vertical motion estimate, becaise the (Jlract btodc matching Is performed on Utnm 
which have twice the numbor of Bnes. 

Concerning consideretion of the final output video fomM. in the ease of the present embodiment the 
converted vMeo is supplied via tape to an electron bem recorder, and needs to consist of frames 
conesponding to the mofion pfeture film rato of 24 frames per second. For this reason, therefore, fiie 

20 pnxtucSon of progiessive scan cofwertad frames is nsoessary, and moreover the progreswve scan 
converted frames can also be used as a falMnck in the case where motion compensated standards 
conversfcm is deemed to be producing unacceptable restrits, for example, where the motion is too diverse to 
be analysed satistectoriiy. in that case the use of the nearest progressive scan converted frame as the 
required output frame can produce reasonably »x»pfable results. 

SB Progressive scan conversion can be caaied out in a number of ways, such as by previous field 
replacement, median filtering in which three spadaly consecutive lines aie examined (temporally tiiese 
three Knes will come from two consecutive fields), or a motton compensated technique which utiiizes multi- 
gradient motkm detectian followed by muttinJirection linear interpolalfcin. However, in the present embodi- 
ment the prefened method is motion adaptive progressive scan conversion, the staps of whtoh are indicated 

ai in the block diagram of Figure 7. The concept is to use interliekl interpoiafion in wholly static picture areas 
to retain as mudi verfical infonnnaHan as possible, and to use hitn-ffeW intarpotaUon when stgnlficam motion 
is present TMs also akis smooth portrayal of mofion. in ecenes where the mofion is somewhere between 
these two extremes, an estimate of the k)cal motton present in the pteture is made, and this is then used to 
mix together different proportions of inter- and intra-fiekl interpolation. 

35 In more detail, the modulus of the frame diflerence between previous and next fields is first generated, 
this being mdfcated in Figure 8. To generate the required estimates, the modukis Inter-frame difference 
array from the prevkxB and the next fleUs te generated 8t each poinfc 

AiKpixei. current Kne. current fieM) = ITQpbal, cunent fine. nextfieU) -Y<pbwl. cunent line, prawtous fieid)| 
where: 

40 /iu is the urHWfmaliiBd modukis (fifferenceanwy, and 
Y is the hminance ancay corresponding to the 30 pldure. 

The modufcis of difference Is then normalized to atUust for the significance of changes bi lower 

An (pixel, current Kne. current field) - F( 7'(pfxel, cunrent line) )* Au(ptxel. current line, cunent fieM) 
45 where: 

Amis the normalized modulus diffeienoe array 
71s the inter-frame average luminance vatoe 

T'JpixBl. current line) » (Y ftjixel. current Dne. prevtous field) + Y(plxel. cunent Hne. next fiekJ) )«. and 
F( 7) (ttie nononaliang function) is derived as imfirated in Figure d. 

so The dIffOrenceanay A Is then verftetfy filtered together with the prevtous fieW difference by a three-tap 
filter (examples of coeffidente are a quarter, a half, a qusrter or zero, unity, zero) to reduce vertical alias 
probleffls. aid in puficular to mMmize the problems encountered with temporal alias. Thus: 
AF{pixei, current Kne. cwront fleW) « AnOwwI. cunent Hne-1. prevkws fiekJyCi + AM(pixel. cunrent Kne. 
current TiMfOz + AN(pixel, cunent llne+1. pievkxis flekl)rCi 

ss where: 

Af is ttie filtered nonndteed dHference array, and 

Ci and Ca are filter coeffidente. and 2Ci +G2 « 1 80 that unity do gabi is maintekied. 

A vented and horizontd totra-fieW filler of up to five taps by fifteen taps is ttisn used to smooth fiie 
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difference values wtthin the current field. In' practice, a flHer W three taps by three taps is sattstadory. 

Rnaliy. in order to produce the actual motion estimation, a non-Dnear mapping funcfion is applied using a 

function to provide the motion estimate (ME): 

ME (pixel, current line) - y (spatialty filtered Af (pixel, cutrert line) ) 
6 The non-linear function y is derived as shown in Rgure 10, the st^c picture ME is zsro, for fuH motion ME 

is one, and for intermediate motions a controlled transition occurs. 

To produce an interpolated pixel, the pixels in the missing One are created by taking proportions of the 

surrounding lines as indicated in i=igure 11. Tlie motion estimate ME is then applied te the rntra-frame 

interpolated value (generated from a two. four, ^ or preferably eight tap filter), and l-ME is applied to the 
10 inter-field average (or alternatively to a more complex Interpolatad value), and theso are summed to derive 

the progressive scan pixel estimate: ^ 

Ym (pixel, cunrent line) « ME (pixel, current Ine) '{ „ , ( y,J(sMi. current Kne-1-2n. cun«nt field) 
+ Y|, ((Afel. current »ne+ 1 +2n, current field) TO^ * d-ME) (Pfxrt. current line) • ( Yh (pixel, cunrent line, 
previous field) + Yh, (pixei, cunent Bne, next fisid) yz 
IS where: 

Co. Ci.C2 and Cs are the intra-frameflHar coefficients, and 2(Co+Ci+C2 -fCs) - 1 so that unity do gain is 
maintained. 

This method of progressive scan conversion is found to produce high quality frames from input fields, 
in particular because a moving object can be isolated and interpoiated in a different manner to a stationary 
30 badcDTOund. 

Rafarring back to FigurB 1, the frames of vkieo derived by the progress scan converter 2 are used to 
derive motion vectors. The esttmatkNi of motion vectors consiste of two stops. RrsUy, cofrelation surfeces 
are generated by conrelating search btocks from consecuUve frames. Then, havii^ ^ined these conrela- 
tion surfaces, they have to be examined to detennlne the position or posittons at which correlation is best 

25 Several different methods of obtaining a con-elation surface exist, the two main methods being phase 
con-elatkx) and direct block matching. There are, however, a number of probtems associated wHh the use of 
phase correlation, these being very briefly problems relating to the transltann mechanism, the windowing 
function, the bkxsk size and the variable quality of the contour of the surface produced. In the present 
embodbnent, therefore, direct btocfc matching is pr^aned. 

30 The defect bkxAmalcher 3 operates as Mkrns. Two btocks, respecfivelyom 

of pixels from consecutive frames of the progressive scan converted video signal are correlated to produce 
a correlation surtece from which a motion vector is derived. 

RefOTing to Rgure 12, firstly a smaH btock called a search btodt of size 32 pixels by 23 lines is taken 
from a frame as shown in l=lgure 12. Then a larger bkwk called a search area of size 128 pixels by 69 lines 

3$ is taken from the next frame. Ttw search bkxk (SB) is then placed in each possible positkin in the search 
area (SA) as shown in Rgure 13, and for each kxatton the sum of the absolute dlfterence of pixel 
luminance tevels between the two btodcs is akaiOsd. This vahje is then used as the height of the 
conrelston surface at the point at whteh it was derived. It can than be used in conjunctkm with other 
similarly derived vetoes tor each possibte kxattton of the search btock to the search area to obtain a 

40 oorraiaiton surfooe. m example of which is shown In Figure 14. Inor clarity the surface is shown inverted, 
and as it is In tect the mkttnum that is required, the required point to Rgure 14 is ttw main peak. 

The size of the search btock is setected by examining the minhnum size of an object that may require 
motten compensation. For PAL 625 lines per frame, SO fiekls per second signals a search block of 16 pixels 
by 8 lines has been found suitable for traddng a small object without allowing any sun-ounding information 

4S not witiiin the object but still within the search block, to affect the tracking of the object This approach has 
therefore been adopted in ttie presem embodiment but nradHted to take account of ttie difterent numbers of 
active pixels per line, active lines per frame, and aspect raOo of a HOVS as compared wHh PAL BZS/SO. The 
comparative figures, the HOVS being put first, are as toltows: 1920 (720) acdve pixato per Hne, 1035 ^75) 
active lines per frame, 3:5.33 (3:4) aspect ratto. 

60 It shoukl be added that ttiere is an argument tor using a larger search Uodt since tills means that a 
large object can be tracked. On the other hand, there exists an argument tor using a smaller search btock. 
to prevent a small otiiect being over-shadowed by ihe effect of a large object or background area. Also, 
however, there is toe advantage that with «nidl search titocks there is no requirement for the deiivalion of 
more than one motton vector from each of them. Because having a single motton vector is so much easier 

55 than having more than one. ttie present embodiment starts with a small search btock as described above, 
and then caiees the seardi btodc to grow into a bigger sean:h btock if no satisfKtory result has been 
obtained. This then encompasses the advantages of both a small and a large search btock. The criteria for 
a saUstectory result is set by the motton vector esttmator 4 (Rgute 1) referred to to more detad betow and 
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which (tetsrnriiMS fi6 motion vBctnr Irani a givBti comlsfion suriaco. 

This tBchraque of causing fhe search blodc to grow Is not only advantageous lor traddng large Directs. 
K can also help ta track fhe movement of an otqect having ttie shape of a regular pattern of a periodic 
nature. Thus, consider Figure 15 where a search blodc A wHI match up with the search area B at locations 

5 VI , V2 and V3, with each of them giving a seemingly correct measure of motion. In this case, however, the 
motion vector estimation, that is the process that actually analyses the correlation surface, will show that 
good correlation occurs in three locafions vrhich are coilinear. The search block will therefore be caused to 
grow horizontally until it is three times its orHjInal widti, this being the direction in which multipte correlation 
occurred in this case. The search area will also be correspondingly horizontally enlarged. As shown in 

TO Figure 16, with the enlarged search block 3A. there is only a single oorrelaflon point, which coirec^ relates 
tothenwtionofthedbieci 

In this particuiar case the search block and flw seardi area both have to grow horizontally , because the 
Erection of multiple correlatkm is horizontal. It is equally pos^e. however, for the search bkxk and the 
search area to grow vertically, or indeed in both directions, tf the correlation surface suggests it. 

IS It shoukl be noted that block matching cannot be applied to all the search bk>cks in tt« frame, because 
in the border area there Is not enough room from which a search area can be drawn. Thus, block matching 
cannot be effected in the border area of the franrte shown hatched in Figure 17. This problem Is dealt wftti 
by the motion vector reducer S (Rgure 1) desolbed in more detail bekw, which tftempts to supply search 
blocks in this hatdwd area with appropriate mofion vectors. 

20 From the oorr^atkxi surface (Figure 14) generaiad for each search tiock in a frame the mothm vector 
estimator 4 (Figure 1) deduces the lawlylntei^ramemolion between the searohbkwk and 
search area It should again be mentioned that for dacity all diagrams of corraiation surfaces shown 
inverted, that is. sudithal a minimum is ^town as a peak. 

The motion vector estimator 4 (Rgure 1) uses motion vector estimation algorithms to detect the 

2S minimum point on each comsiation surface. This represents the point of maximum correlaiion between the 
search block and fhe search area, and heince indkates the probable motton between them. The displace- 
ment of tWs minimum on the comMon surtsce wift respect to the origin, in this case the centre of the 
surface, is a direct measurement, in terms of pbieto per frame, of me motion. For the Amplest case, where 
the corr^atton surtoce contabis a single, dbfinct minimum, the detection of the minimum point on the 

30 coneiaSonsurtee is sullteiert to determirie accurately the motion between theses 

area As preykMisly mentioned, ttie use of small search bkicks improwes tiie detection of motion and the 
accuracy of motion estimatkia but unfortunately smatt single search btocks are unable to delect m^ 
number of drcumstances whteh will now be described. 

Rgure 18 shows an o^ect witt> motion vectors (5. 0) straddMng fliree search bkjcks 1A. 2A and 3A In a 

3S frame (t).VVhen the searcJibhwksiA and 3A are correlated with respective search areas (IB and 38y in tto 
next frame (t+1) a conflation surface shown in Rgwe 19 resutts showbig a minimum st (S. 0). (This 
assumes a noiseless vkteo source.) However, when the search bktck 2A is corralattd with Hs respective 
search area 2B, the correlation surface shown in Rgure 20 is produced, in which the seerch btock 2A 
correlates with the search vea 2B at every point in toe y«d8 direceon. There is ttierefore no singN 

40 mMnwm in ttie connelation surface, and hetMflw motion between tin search btock 2A 

However, now conskter the situation if the search btock 2A is grown such that it encompasses all ttiree 
of toe original search btocks lA. 2A and 3A. When flie grown search btock 2A is correlated with a search 
area covertog the original search areas ia 2B and 3B. the resulting correlation surface is as shown in 

45 Rgure 21. Thte shows a single minimum at (5, 0) indicating tt» correct motion of the original search bkwk 
2A. This exampte illustrates ttie need for some unkjue feature in ttw source vWeo. in order accurately to 
detect motion. Thus, the search btocks 1A and 3A bottt had unkyie vertk»l and horizontal features. ^ 
the edges of ttie object, and hence motion couU be determined, to cOnHast. ttie search btock 2A had a 
untoue vertical feature, but no unkyje horfaxfrtal taature. and hence horizontal motion couM not be 

SO detemunsd. However, by growing ttie search btock until H encompasses a unique feature botti horizontally 
and verttcaHy. ttie complete motion for that search blodc can be determined. Moreover, K can be shown that 
growing ttie search btock is beneiteial when notse in ttie source video Is considered. 

A torttier exampis win now be considered vritti reference to Rgure 22. This shows a conrelation surface 
for 8 search btock where the motion vector is (5. 3). However, due to fhe numerous other correlattons which 

55 have taken place between ttie search btock and ttie search area, the true motion is difficult to detect. An 
ffiompte of soiHce vkleo whfch might produce such a correlaBon surface wouki be a tow contrast tree 
moving vrito tt» wbid. ft to now assumed ttiat ttie seaich btock and ttie search area are growa The growing 
can take place in ttie horbontel drection. as in the prevtous exampte. or to ttie vertical direction, or to botti 
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directions. Assuming that the neightxxiring search tjlocks have the same motion, the mean effect on the 
resulting correlation sur^ will be to increase the magnitude of the minima at (5, 3) by a greater proportion 
than the magnitude of the other conflation peaia. TNs is shown in Rgure 23, whid) hdicates that it is then 
easier to detect the conrect motion vector. 

5 The way in which search blocks are grown wfli now be fiirther considerecl with reference to Rgure 18. 
Here it was required to grow the area of ttie search block 2A to encompass the areas of the search blocks 
1A and 3A. and to produce the resuNing correlaikxi surtace. bi feet the resulAng correlaiion surfaces are 
produced directly by adding together the elemeats of the three oonelalion surfaces conresponcMng to the 
search bkwks 1A. 2A anJ 3A. In effect. If each conelatk>n surface is consklered as a matrix of point 

10 magnitudes, then the correiadon surface of the enfauged searcft block 2A is Ihe matrix addition of the 
conetaOon surface of the original search bkicka 1^ 2A and 3A. 

The area of the search btock 2A could also be grown verticaliy by adding correlaton surfaces of the 
search btoOa above and bekwr, whilst if the search block 2A is to be grown botti horizont^y and vertically, 
then the four neighbouring diagonal correlatkxi surfaces have to be added as well. From this it will be seen 

rs that the actual process of growmg a search btock to encompass neighbouring search btocks is relatively 
easy, ttie more difficult process being to deckle when growing shouk) take place, and whteh neighbouring 
search bkicks should be encompassed. Basically^ ihe answer is that ihe area of the search bloda should 
be grown unfll a good minimum or good motkM) vector is detected. It Is therefore necessary to spedfy when 
a motion vector can be taken to be a good motkm vector, and this can in fact be deduced from the 

20 examples given above. 

In the example described with refisrBnce to Figures 18 to 21, it «m necessary to gnnr the 
horizontally in order to encompass a uniQue lx)rizanlai feature of the object, and hence obtain a single 
minimum. This situatkm was characterized by a row of kientteal minima on ttie correlatton surfece of Figure 
20. and a single minimum on the correlation surface of Figure 21. From this the first criteria for a good 

25 minimum can be obtained; a good minimum is the point of smallest magnitude on the correiatton surfece 
tor which the difference between it and the magnitude of the next smallest point exceeds a given value. 
This given value Is known as the threshoU vakie. and hence this test is refenred to herein as the tiireshoM 
test 

It shoukl be noted that the next smallest point is prevented from ori)^nating from wHhin the bounds of a 
30 furthw test, descrttwdbehm, and ratarrBd to herein as the rings tesL in the case of a r^ 

three rings, the next smallest point is prevented from originating bam a point within three pixels of the ponit 
in quesfion. in the example of Figures 18 to 21. the csrrelatkm surtace of Figure 20 wouM have failed the 
threshold test; the search area 2A is therefore grown and, given a suitable thretiioM vakie, the correlation 
surface of Rgure 21 will pass the threshold (est 
35 The ttireshold test can also be used to cause growing in the example described above with reference to 
Figures 22 and ^. Prtor to growing flie search triock. the conrect minimum is undetectable, due to the 
ck)sely similar magnitudes of the sunounding points. Given a suitabie threshold vahis, however, the 
correlation surface will fail the threshokl test, and the search bkx* will be grown. AS a resuil it 
possible to detect the minimum anwng the other spurkws points. 
40 It wil be seen that «ie use of a ihieshoM is a subiecthn test, but the correct threshoM for the 
correlation surface under test can be setodsd by nomnaRzing the threshold as a tacSOn (tf the range of 
magnitudes within the coneiatkin surface. This also lessens the eftact of, for example the contrast of the 
vkteo source. 

The rings test, refenred to briefly above, and whteh is far le» subjecfive. will now be further described. 

45 The basis of the rings test is to assume that a good minimum (or maxinuHn) wiH have points of increasing 
(or decreasing) magnitudes sunounding it Figure 24 illustrates this assunptkm. showing a minimum at (0, 
0) where the surrounding three rings of points hsMtt decmasing mean magiiitude. litis Is as opposed to the 
conelation surface shown in Figure 25, where the rings, and in particular fte second bmer^nost ring, are not 
of decreasing mean magnitude. 

SO tn this case the criteria for a good minimum as defined by the rings test, is that the average slope is 
monoionie. Therefore for a pre-ddined number of rings of points surrounding ihe minimum in question, the 
mean magnihide of each ring when moving ftom the Innermo^ ring outwards, must be greater ttian mat of 
the prevkxis ring. Fletuming again to the example described with reference to Figures IB to 21, it will be 
seen from Rgures 20 and 21 that 0ie correlaikxi surface of F^ure 20 wouU have faikxl the rings test, but 

55 that the correlation surface of Rgure 21 v»ouM have passed the rings tesL Since the rings tost compares 
mean, and not absokite. magnitudes, it is far less subiecdve than the thrMhoU test, and kidaed the only 
variable m the rings test is the number of rings oonskiersd. 

leaving descftt)ed Hie meclianism for growing a search bkxdc it is now necessary to consider how by 
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examining tiie shape of ttw correlation suitace it Is passible to datamine the most effective drection in 
whl(^ the search block shoild grow. 

Referring a^'n to Rgure 20. tfiis corr^dion surface rssuited where there was a unique vertical feature, 
but no unicpje horizontal tenure. This is mirrored in the correlalion surface by the minimum running 
s horizontaUy across «» correlation surface, due to the multipie conelations In this direction. From this It can 
be deduced that the seand) block should be grown horiaxitally. Conversely, ^xxikl a 8ne of multipie 
correlations run vartieally, this would indicale the need to grow the seaicli block vertk»lly, whilst a circular 
collection of mufliple coiralatkms would indkato a need to grow the search bkick both horizontally and 
verticaliy. 

TO Using this criteria, a quantatlve measure of the shape of the corielatkjn surface is 1^ 

determine in wliich direction the search block should tie grown. This measure is determined as tollows. 
Firstly, a threshoM Is determined. Any poftit on the correlstion surtece bek>w the threshoU is ttien 
co(«sidered. This thresiwkl. Bke that used In flie ttweshold test, is nomudteed as a fracfion of the range of 
magnitudes within the correlation surfiace. Using this fhreshokl, the points on the correlation surtoe are 

IS examined in turn in lour specific sequences. In each, the point at whbh the correlalion surface value fails 
bekMv the threshold is noted. These four sequences are illustrated diagrammaitoally in Rgure 26 in which 
the numbers 1. 2, 3 and 4 at the tD(>. bottom, left and right refer to ffie four sequences, and the hatched 
area indicates points which fall below the threshold: 
Sequence 1 

20 Search from the top of the correlalion surfsoe down for a point A whtah faUs bekiw ths thrsshoU. 
Sequenoea 

Search from the bottom of the correialkin surfece up lor a point C whteh falls betow the thr^hokL 
Sequences 

Search from the left of the correlation surface to the right for a point D whtoh fate below the threshoki. 
2S Sequence 4 

Search from the right of the coneiafion surface to the left for a point B which falls below the threshoki. 

The k>cations of the four resulting points A, B, C and D are used to calculate the two dimendons X and 
Y indicated in Figure 26, these dimensions X and Y indicating the size of the hatched area containing the 
points falRng betow me threshoki vahje. Hence from the dimensions X and Y, It can be deduced whether 

30 the shape is k>nger in the X rather than the ydireclkM, or vk» versa, or whether the shape is approxim^ 
drcidar. A margknl dHference of say ten percent is ^Howed in dedudng the shape, that is, the dimension X 
must be a minimum ol ten percent greater Urn the dmenston Y for the shape to be conskiered to be 
kxiger in the x direction. Simiierly for the y diredion. If the dbnenskms X and Y are within ten percent of 
each other, then the shape is considered to be a'reular, and the search bkick is ^own in both directkms. In 

36 the example of Figure 26 the dinnenslon X is grealBr than the dimension Y. and hence the search block Is 
grown in the x or horizontal direction. 

The growing of the search bkick continues unfit one or more growth limttattons is reached. These 
limitations are: that the minimum in the correlafion surface passes both the threshoki test and the rings test; 
that the edge of the vkJeo frame is reached; or that the search btock has already been grown a 

40 predetermined number of times horizontally and vertically. TMa test Hmilatkm is hardware dependent That 
Is to say. it is Smited by the amount of processing that can be done bi the awallaUe tbne. ki one spedfk: 
embodiment of apparatus accordtog to the present hiventton. this UmK was set at twk» horizontally and 
once vertically. 

If the minimum in the correlatton surface passes both the threshoki test and the rings test, then it is 
■4S assumed that a good motion vector has been deiermbied, and can be passed to the motion vector reducer 
5 (Figure 1). However, if the edge of the Same is reached or the search btock has already been grown a 
predetermined number of times both horizontally and vertkally. then it is assumed that a good motkm 
vector has not been deteimlned for that partteuiar search block, and faistead of attemplhig to dslennbw a 
good moSon vector, ttie best available vector is determined by wtighHng. 
50 The conelaikin surtoce is weighted such that the aOoaexin of tfw best avallabte motkm vector is 
weighted towwds the siaBonary. that Is the oenbe, motton vector. This is tor two reasons, firstly, if the 
search bkx*. even after growing. Is part rf a large ptain area of source wdeo, it *i«l not be possible to 
detect a good motion vector. However, since the source video Is of a pUn area, a stetionary motion vector 
will lead to the correct results in the subsequent processing. Secondly. welgMkig is designed to reduce the 
5S possibility of a seriously wrong motion vector being passed to the motkm voclor reducer 5 (F^ure 1). This 
IS done because it is assumed that when a good mofion vector cannot be determined, a small inoonrect 
motion vector is preferebki to a large incorrect motion vector. 

Rguie 27 shows an example of how the weighting ftmcfion can be applied to the conelaikjn surface, ki 
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this oampto. the weigM applied to a given point on the oonelatldn surface is diret^ proportional to tfie 
(finance of that point from the idafionary. oonlre vector. The magnitude of the point on the 

conrelation surface miritipfled t>y (he weighting feKrtor. For example, the gradient of the weighting function 
may be such that points plus or minus 32 pixels from the centre, stationary motion vector are multiplied by 
5 a factor of three. In other words, as shown in Figure 27, where the centre, stationary motion vector is 
indicated by the black circle, the weighting fuhction is an inverted cone which is centred on the centre, 
stationary motion vector. 

After the correlation surface has been weighted, it is ^aln passed through the threshold test and the 
rings tasL If a mininHim which passes i»th ffMse tests is determined. «ien it is assumed 

10 nnoiion vector, and it is flagged to indicals that it is a good motion vec^ 

flag is passed, togettwr with the motion vector to flw moBon vector reducer 5 (Figure 1). If on the other 
hand, ntfiher a good mofion vector nor a best available mofion vector can be determined, even after 
weighting, then a flag la set to indicBto that any motion vector (nssed to the motion vector reducer S 
(Figure 1) for this search btock is a bad motion vector. It is necessary to do this because bad motion 

*s vectors must not be used in the motion vector reduction process, but must be sub^tutsd as win Iw 
described below. 

Thus, in summary, the operation of tfje mofion vector estimator 4 (Figure 1) is to derive from the 
conflation surface generated by the direct block matcher 3 (Rgure 1). the point of best con'elation, that is 
the minimum. This minimum is then sid^ected to the threshold test and the rings test botti of which the 
20 minimum nwst pass in order for it to be oinsiderad to represent the motion of the search bkx:k. Ms 
incidBntally, be noted that the thrertnU used in the threshoM test and the rings test may be either absolute 
values or fractional values. If tlw minimum falls either test, 4ien the search block is grown, a new minimum 
is determined, and the threshokl test and the rings test re-applied. The most effdc^ direction in which to 
grow the search bkick is determined from the shape of the corrslatkxi surface. 

35 Retoning initially to Rg^e l, the process of motion vector reduction will now be described. Using a 
HDV8, each search block is assumed to be 32 pixels by 23 lines, whk^ can be shown to lead to a possible 
maximum of 2451 motion vectors. The choice of the search btock size is a compromise between 
maintaining resolutton and avokiing an excessh« amount of hardware, if all these mofion vectors were 
passed to tite motion vector selector 6, the task of motion vector setoction would not t)e praclicat>lei due to 

30 the amount of processing thrt would be required. To ovsioome this pfDUem, the moGon vector reducer 5 is 
provided t)etween the motion vector estimator 4 and the mofion vector selector 6. Tlie motion vector 
reducer 5 iaices the motion vectors that have been generated tiy ti)e moUon vector e^imator 4 and presents 
the motton vector selector 6 with only, for example, four moOon vectors for each search block in the frame, 
Including those in border regkins, rather than all the motion vectors derived for that frame. The effect of this 

36 is two-fold. Firstly, this makes it much easier to choose the correct motion vector, so tong as It is within the 
group of four motfon vectors passed to the motion vector selector 6. Secondly, however, it also means that 
if the coned motton vector is not passed as one of the four, then the motion vector selector 6 is not able to 
select the correct one. It is therefore necessary to try to ensure that the motton vector reducer S Includes 
the correct motfon vector amongst those p^sed to the molton vector selector 6. It should also be 

40 mentioned that although four molkm vectore are passed by the motion vector reducer 5 to the motton 
vector setoctor 6, only three of these actually represent motion, the fourth motion vector always being the 
staltonary motion vector whtoh is indudad to ensure that the motion vector selector 6 is not forced into 
applying a moticm vector r^wesenting motion to a stationary pixel. Otiwr numbers of motion vectors can be 
pinsed to the motion vector setoctor 6, for axampte. In an aHemattve embodiment four motion vectors 

45 representing motton and tite stationary motion vector may be passed. 

Hsretoafter the tenn 'sample btock' refers to a btock in a frame of vktoo in which each pimi is offered 
the same four motion vectore by the motion vector reducer S. Thus, a sampto btock is tite same as a 
seartii btock before the search btock has been ^own. As shown in Figure 28. in a finme of video the initial 
positions of the sampto btodis and the search t}loclcs are ifie same, 

SQ The nwtton vector reducer 5 (F^jure 1) receives ttw motion vectore and. the flags from tile n^ 
vector estimator 4 (Figure 1) and detenmines flie quality of the motion vectore by examining the flags. If tite 
motion vector was not derived from an ambiguous surface, ttiat is there is a high degree of confides 
then It Is termed a good nwtion vector, but if a certain amount of ambiguity exista, than the motion ve^ 
tonned a bad motion vector. In the motion vector reduction process, all motion vectors classed as bad 

K motion vectors ore ignored, bocauae it is miportsnt tint no inaarrect motion vectore ore ever passed to tite 
motion vector sdector 6 (Figure 1), in case a bad moDoii vector is seiacled thereby. Such eetection would 
generally result in a spurioiis dot m the final picture, wttich wouM be h^y visHile. 

Each of tiw motton vectore suppHed to the motion vector reducer (Figure 1) ws» obtakwd from a 
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particular search block, and hence a particular sample block (Figure 28). the posifion of these being noted 
«Dgether with the motion vector. Because any motion vectors wWch have been ctassed as bed mofion 
vectors are Ignored, not all sample blodcs will hawe a motion vector derived from the search blocic at that 
position. The motion vectors which have been classed as good motion vectors, and which relate to a 

5 particular search biodc and hence a particular sample block, are called hxal mofion vectors, because they 
have been derived in the area from vrtiteh the sample bhx* was obtained. In addition to this, anothw motion 
vector reduction process oounte the frequency at whk:h each good motion vector occurs, with no account 
taken of the actual positions of the search bkicks that were used to derive them. These motkxi vectors are 
then rankBd bi enter of dc .:rn^ frequency, and are called common motion vectors. In the worst case 

TO only three cornrnon molion vectors are amilable and these are co^ 

to make up the four motkm vectors to be passed to the moiton vector selector 6 (Figure 1). However, as 
there are often more than thrse common motion vectors, the number has to be reduced to fonn a reduced 
sat of common motion vectors referred to as gtobel motton vectors. 

A simple way of reducing the number of common mothm vectors is to use the three most frequent 

7S corhmon motion vectors and disregard the remainder. However, the ttiree most frequent common motion 
vectors are often those three motion vectors which were inifialiy wHhki phis or minus one pixel motion of 
each other verficaOy and/br horizontally. In other words, these common motion vectors were a! traddng the 
same motion with sSght differences between them, and the other common motkm vectors, whtoh wouU 
have been disregarded, were actually traddng different motions. 

20 fri order to select the common inoflon vectors wfiich rsprssent aO or most of the motion in a scene, it is 
necessary to avoid choosing glot>al motfon vectors which represent the same nxjtion. Thus, the strategy 
achislly adoptsd is first to take the three most frequently occurring common motkm vectors and chedc to 
see if the least frequent among them Is wiflihi plus or minus one pixel motkm vertically and/or plus or minus 
one pixel motkm horiamtally of either of ttie other two common motkm vectors. If it is. then it is rejected, 

25 and the nod most frequently oocunring common motion vector is diosen to replace it This process is 
continued for ail of the most frequently occurring common motkm vectors until there are either three 
common motion vectors whk:h are not similar to each other, or until there are three or less common motion 
vectors left However, if there are more than three common motion vectors left, then the process is repeated 
this time checking to see if the least frequent among them is within plus or minus two pixel motion verticaiiy 

3D and/br phis or mimis two pixel molon hori2»nialiy of anottwr, and so on at increasing distances if 
necessary. These three common modon vedorS'Srs the required glotxi motion vectors, and it is important 
to note that thqr are stin ranked in order of Crequency. 

When considering the motion vector redudfcm process and the sampto bk)d(s of a frame of video, it is 
necessary to look at three different types of sample blocks. These types are related to their actual position 

3S in 3 ftame of video, and are shown in F^re 28 as regkms. Regton A comprises sample t>tocks whtoh ere 
totally sunounded by ottier sample blocks and are not near the picture boundwy. Hegton B contains sample 
blocks which srs paitiaily sufrounded by other sample btocks and are not near the picture boundsy. 
Finally, regton C contains sampte blocks whlcii are near the picture boundary. The mofion vector reduction 
algorithm to be used for each of these regions is different These algorithms wiB be descrHMd betow, but 

40 firsily it shoukl be raitaralBd that there «dst good motton vectors for some of the stgnple bkx;fcs In the 
frame of vMeo, and additionally there are aiso tluee gtotial motion vectors which should represent most of 
the prsdomlnant motton in the scene. A selecflon of these motkm vectors is used to pass on ttvee mofion 
vectors togettwr with the stalkmary ntotkm vector for each sample btock. 

Figure 30 ilkistrates diagrammattaally moiton vector reductkm in the region A. This is the most complex 

45 regkm to deal with, because it has the largest number of motton vectors to check. Figure 30 shows a central 
sample btock which is hatched, sunounded by other sampto btocks a to h. Firstly, the locally derived 
moiton vector Is examined to see tf it was classed as a good mofion vector. If It was. and it is also not ttw 
same as ths stafionary mofion vector, fiwn it is passed on. However, if It fSls eittwr of these tests, H is 
ignored. Than the motkm vector associated with the sampte btock d is checked to see if it was classed as a 

so good motton vector, if it was, arxl if it is neifiier the same as any mofion vector already aeiacted, nor the 
same as the stationary motion vector, then it too is passed on. If it Ua\s any of these tests then it too is 
ignored. This process then continues in a similar manner in the order e, b, g, a, h, c and t As soon as three 
mofion vectors, not inckiding the stationary mofion vector, have been obtained, then the algorittim stops, 
because that is all ttwt is required for mofion vector selection for ttiat sample block, tt is. however, possible 

S5 for all the above checks to be carried out wifiwot ttwee good motion vectors having been obtained. If tt»is is 
fiw case, ttwn file mnatoing spaces are tilted witti ttw gtobal mofion vectors, wittiprtority being given to the 
more frequent gtobal mofion ve^MS. 

Figure 31 illustrates mofion vector reducfion in fiie regton B. Sampte bk)ck8 to ttie regkm B are ths 
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same as those in tie region A. except that they are no« totally su^^ 

process applied to these sample lilocte is exactly the same w those for ttie region A. except that it is not 
possible to search in all the surrounding sample blocks. Thus as seen in Figure 31, it is only possible to 
chedt the motion vectors lor the sample blocks a to e. and any remaining spaces tor motion vectors are 

s filled, as before, with global motion vectors. Utewise, if the h^hed sample btock in Rgure 31 were 
(fisplaced two positions to ttie left, then it will seen that tfiere would only be tfiree adjacent surrounding 
blocks to be checked before resorting to gtobal motfcm vectors. 

Figure 32 iilustratas motton vector reductkxi in the region C. This is the most severe case, because the 
sample tkxks neither hame a kically d»ived moQbn vector nor do they have many sunounding sample 

10 bk>cks whose motion vectors coukJ be usad. The simplest way of dealing witti this problem is simply to 
give the sample titocks In the regkm C the gkibal motion vectore together with the stalionaiy motion vector. 
Howevw, this is found to produce a blocks eftsct in flie resulting picture, due to the sudden change in 
the motton vectors presented for the sample bkwks in the region C compared with adioining sample btodcs 
in the region B. Therefore a preened strategy is to use for the sample blocks in the regkm C the sample 

)5 motion vectors as those used for sample btocfcs in the regton B, as this prevents sudden changes. 
Preferably, each sample btock in the regnn C is assigned the same motton vectors as that sample btock in 
the region B whtoh is phystoally nearest to it. Thus, in the example of Figure 32. each of the hatched 
sample blocks in the regkm C wouU be assigned the same mothxi vectors as the sample btock a in the 
region B. and this has been found to give exeellem results. 

20 Tills assignment of mofloit vectors to the sampto btocfcs bi the region B, wHh wtiich the present 
invention is partkailarly. but not exclusively, concerned, will now be described in more 
to Figures 33 to 35. 

Figure 33 shows a circuit arrangement lor sHiBcling the requirsd usigranent in the region B. The 
arrangement comprises a count system Including counters 21 to 24. buffers 25, 26, 28, 29, 32 and 33, a 
25 microprocessor 27, s random ac ce ss memory (RAM) 30, and a read only memory (ROM) 31 which holds 
the program for the microprocessor 27, these eiemenis being interoonnecled as shown and operating as 
will now be described. 

The count system 20 is reset at the start of a frame and then tiie moHon vectore for ttiat frame are 
toaded by way (rf the buffer 32. WNto the motton vectore are being loaded, the mtoroprooessor 27 is 
30 disabled, as it is also whBe the motton vectors sresutMquetfliybeiiHi read out by way d the buffw 

White the molton vectore are being toaded, 0to oountsr 21 counts the inpu motton vectore, and hence 
controls the input of the nwtton vectors to the RAM 30 by providing sample btock addresses by way of tt^ 
txiffer 25. Meanwliile, the counter 23 couits Hie aampte blocka in the horizontal directton of the frame, aiKl 
the counter 24 counts the sampto btocks In the vertical dtraclton of flie frame. On comptotton of the frame, 
as the counter 24 causes the eounlsr 22 to control the output of the moOon vectors from the RAM 30 by 
providing sampto btock addresses by way of the buffer 26. 

The RAM 30 has three areas. A first area is set askto to store the original motton vectore relating to the 
sample btocks. This area will only be read by the mtoroprooessor 27, and will never be written in by the 
microprocessor 27. A second area is set askle to store the output molton vectors, as this makes reading out 
40 easier. A third area Is set asidd for processing. 

Initially then the mtoroprooessor 27 and the moBon vector output section are disabled wMto the 
incoming molton vectors are written to ttie RAM 30. Tlie motton vector input and output secttons are then 
dtoabM white processing accordtog to flw algorithm descrtoed betow is pettormed. Then the motton vector 
input sectton and the microprocessor 27 are disabled white the motton vectore are read out of the RAM 30 
45 by the motton vector output section by way of the buffer 33. 

RefMig to Figure 34, this shows a frame of'vftjeo, and to parttoular sample btocks conesponding to 
the region B, that Is the sample btocks in the region immediately adjacent to but not actually In the border 

Fir^ of all constoer the sampto btock a. If the tocal molton vector mtt has been generatod for the 
50 sampto btock a is botti good and non-zero, then it to retained as one of the four motion veeiore for the 
sampto i)tock a. Next, tlia local motton vector generated fbr the sampto t>iock d to comparad with that 
already retained for fte sampto btock a. If It to the same as ttie toctf motton vector for the sarnpto btock a or 
to zero then it is discarded, ftowever. If it to good, non-aero and not the same as the tocid motion vector for 
the sample block a «ien it too is retained. Then ttw tocal mofion vector for ttw sampto btock b is compared 
ss vriththosealready retained tor the sampto btock a. If it to the sanra as a motton vector already retained or is 
zero It to discarded, iiowever. If It n good, non-zero and not the same as a motkm vector aki^^ 
too to retoined. Then the toed motion vector for the sampto trfoek c to exaintoed to a simitor way, but in thto 
case against the tocal motton vectore tor ttie sampto btocks a. d and b if ttiey have been retained. 
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The remaindsr of the sample blocics sKgaoent to the border mfion C am moliain vector reduced in a 
similar way. However, the sample blocks used tor the seleciion of motfon vectors to be retatoed depend on 
the location of the original sample block relaSve to the border region C. The foloiring lists exanMes of the 
various different kicattons that the algortthm has to (tod with, and In eacAi case shows the sampto bkicks 
5 consMered. and tf» order of this consMerafion. 
Sample block a : sample btocks d, b, c 
Sample block e : sample btocks f. h. g 
Sample bkx:k q : sample bkKks r, f. s, t g 
Sample block u : sample btocks v. h, w. g. x 
TO Swipis block I: sample blocks I. j.k 
Sample btodk I : sample blocks y. i, k. z, j 
Sample block m : sampia blocks p, n, o 

Motion vector reduction stops if eittier three motion vectors have been retained, or there are no sample 
btocks left to be examined, if three motion vectors have not been retained for a sample btock, then the 
IS remaining spaces will be filled by using the gtobal motton vectors in order of frequency. In this way each 
sample block should get three different non-zero motion vectors phis the zero, that Is the siattonary motton 
vector which is always supplied. 

The operatton of the arrangement of Figura 33 in accordance with this algorithm is shown in the form of 
a ftow chart in Figiro 35. 

so Rstoning again to Rgure 1, the puiposs of Ih6 motkxi vactor salector 6 is to assign ons of Itis four 
mofion vectors suppled thereto to each indhridual pixel within the sampto btodc bt Ms way the motion 
vectors can be correcUy mapped to the oulflne of Obiocts. The way In which this assignment is effected is 
parttoulariy intended to avok) the possibility of the background sumxinding fine detail from producing a 
better match than that produced by the correct mtrtton vector. To achieve this the motion vector selection 

26 pnxess is split into two main stages, in the first stage, motton vectors are produced for each pixel in the 
input frames, in other words, there is no atterr^rt to determine the motion vector vakjes for pixels at the 
output frame positions. The second stage uses the mofion vector values produced by the first stage to 
determine the motion vector value tor each pfocel in tho output frame. 

RefMng now to Hgim 33, each pixel of the Input ftame 2 is tested for the best luminance value 

30 with the pravkxis and foltowing input finmes i and 3 of vktoo data, using each of the fbur mom vectors 
supplied. The pixel lumkianoe dtffbrence is detannined as: 



^ Pim, is ttie luminance value of a frame 1 pixel within a 4x4 btodt of pixels sunroundtng the pixel whose 
tocation is obtained by sutrtracting the coordinates of the motton vector being tested from the tocsXxsn of the 
pixel being tested in frame 2 

P2«„ is the luminance value of a frame 2 pixel within a 4x4 btock of pixels surrounding the pi:«i being 
te^ed 

45 P3n„ is the luminance value of a frame 3 pixel within a ^4 btock of pixels surrounding the pixel whose 
tocation is obtained by adding the ooordinales of the motton vector being tasted to the tocatton of the pixel 
being tested to frame 2 

The minimum pixei dinerence then indk»tos the best kmrinanoe match and thacefbre t^ 
vector appRcabto to the pbeei b^ tested. If the conect mofion vector is not avaHabto, or there are 

gg uncovered or covered arms, referred to in more delaHbetow, then a good match may not occur. 

The indcdion of a poor match Is achieved when the average pixel difference within the bkxk of pixels 
being used is above a certain threshokl. This threshoM is important because high frequency detail may 
prodis» a poor match even when the correct motton vector is tested. The reason for this poor match is the 
possibility of a half pi)«t mar in the motion veoor estimate. To determine what threshoU shouM indicate a 
poor match, it Is necessary to relate the threshoU to the frequency content of ttie picture witttin the block of 
data whtoh surrounds the pixel for whtoh the motion vector is required. To achteve this, an autiHhreshoM 
vakie to detannined where the threshokl vakie eqtate m flw maximum horizontal or varticai pixsi 
himinance difterence about ttie pixel being tested. To ensure that the threshokl value obtained is 
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repres6niBliv9 of Ihs whols block of dsia wtiich is oompvwlt wi avorego valu6 is obtainsd for the four 
central pixels of a 4x4 Uocic used. 

Referring to Figure 38, which shows a 4x4 block, the requhvd thresholcl value T is given by: 
T = (T1 + T2 + T3 + T4)/8 

5 where T3. for example, is delBrmined as intScatBd in Figm 38 as equal to the maximum of the four 
himinance difference values comprising: 
«ie two vertical differences |B2-B3|and|B4-B3|,and 
tfie two horteomal d»ferenoes j A3 - B3 1 and I C3 • B3 1 

In this way a frame of motion vectors is obtained lor input frame 2. and in a similar manner a frame of 

10 motion vectors is obtained for inputframe 3 as MicalBd In Figure 37. 

Apart from scene changes. It is the phenomenon of unooveradAxwered surfaces that causes a mfs- 
match to occur in the above first stage of motkxi vector seleclton. If an object say a car, drives bifo a 
tunnel, then the car has become covered, while when it drives out, the car is uncovered. If the part of the 
car that was uncovered in trames 1 and 2 is covered in frames 3 and 4, ttten the basic vector selection 

ts process Is not able to detemtine the con«ct vector. Moraovar, whilst the car going into the hmnel becomes 
covered, the road and objects behind the car are being uncovered. Likewise the car leaving the tunnel is 
being uncovered, but the road and otqects behind the car are being covered. In general therefore both 
covered and uncovered objects wiU exist at the same time. The end of a scene will also have a 
discontinuation of motion that is similar to an object becoming covered. In an attempt to detormine a motton 

20 vector even in such circumstances, the luminanoe ^oiue blocii match Is reduced to a two frame match, 
instead of the three frame matoh of Figures 36 and 37. Tlie frame that the mofion vectors are required for 
(say frame 2) is block-matched indhridualiy to the prevtous and the next frame (frame 1 and frame 3 
re^wctively, in fte case of frame 2), u^ng the four motfon vectors supplied. The motion vector whfch 
produces the best match is chosen as the motton vector applicable to the pixel being tested, in this case, 

25 however, a flag is set to indk»rte that only a two frame match was used. 

Particulariy with integrating type televiskm cameras, there will be situations where no match occurs, if 
an ot^ moves over a detailed background, then an integrating camera will produce unk|ue portions of 
picture where the leading and fraiUng edges of the object are mixed wflh the d^T of the background, in 
such circumstances, even the two frame match couM produce an average pixel difference above the 

90 threshoki value. In these cases tt» motion vector value is set to zero, and an error flag to aim S8^ 
The second staoe of motton vector selectton makes use of the two frames of motton vectors^ 
the first stage. One tome of motton vectors Onput frame 2) is consktoied to be ttto rafOi^^ 
foltowing frame to this Cmput flr»ne 3) is also used. The output frame position then exists swnewhere 
between ttiese two frames of motfon vectors, f^ening to Rgure 40. for each output pixel positton the four 

3S possible motion vectors assodatod with the sample btock of input frame 2, are tested. A line drawn through 
the output pixel position at the angle of the motion vector being tested will point to a posifion on both the 
input frame 2 and the input frame 3. in the case of odd value motton vectors, for example, 1 , 3 and 5, a 
point midway between two input frame pixels woW be Indteated In the case where the output frame is 
precisely half way between the input frames 1 and ^ To altow for ttds Inaccirocy. and also to reduce the 

40 sensilivtty to hiCMdual pbnis. a 3x3 bkxdc of moBon vectors Is acquired for each frame, oenfred on toe 
ctosett pixel positton. In eHect a btock-malch is then pertomned between each of the two 3x3 btocks of 
motton vectors and a block containing the motion vector iMhig tasted. The motion vector dRfsrance used 
represents the spafial difference of the two motion vector vahies as given by: 
Vrx1-x2J* + (y1-y2« 

x1 and y1 are the Cartesian coordinates of the motion vector In one of the bfocks 

x2 and y2 are the Caitesian coordinates of the motion vector being tested 

An average vector difference per pixel is produced as a result of the btock match. 

A motfon vector match Is firet produced as above using only motton vector values whfoh were 

50 cakailatod using three input frames; that to. input frames 1.2 and 3 for toput frame 2 (Figure 3^^ 

frames 2. 3 and 4 for input frame 3 (Figure 37). and the result is scaled acoorcBngly. Prafoiably there are at 
toast four usabto motton vectors In the btock of nbie. When both the motton vector block of frame 2 and 
frame 3 can be used, the motton vector difforenoe vatoes are made up of hedf the mofion vector difference 
value from frame 2 phis half the nwtton vector dHlteence value from frame 3. Whichever motton vector 

95 produces the minimum motion wcux dMsience value uatog^the above techraque is constelared to be the 
motton vector applcabto to the output pixel being tested, tf the motton vector difference vahie produced by 
the three ilnme match input nmiton vector (Figures 38 and 37) is greator than unity, then a covered or 
uncovered surface has been delected, and itie same process ie repaatsd, but tMs time Ignoring the error 
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flags. That is. the motion vector values which were calculated using two input frames are used. Theoreti- 
cally this is only necessary fbr uncoveiedteovered surfaces, although in fact improvements can be obtained 
to the picture in more general areas. 

If after boUi of the above tests ham been performed, the minimuffl motion vector match is grsaier than 
5 two. the motion vector value is set to zero, and an enror flag is set for use by the motion vector post 
processor 7 (Rgure 1). 

Following motion vector selection, there wiO almost certainly be in any real picture situation, some 
remaining spurious moHon vectors associated with certain pixels. Figures 41 to 46 show what are talcen to 
be spurious motion vectors, and in eadt of these figures the triangles represent pixels having associated 
10 therewith the same motion vaclors. whHst the sters reprssent pixels having associated therewith motion 
vectors dHfeient those associated with the surraundbng pixels, and the drcte indicates the mottm vector 

Rgure 41 shows a point singularity where a single pixel has a moUon vector diffsrent from those of aH 

the sunrounding pixels. 

fs Figure 42 shows a horizontal moflon vector impulse, where three horizontally tdigned pixels have a 
mofion vector different from those of the surrounding pixels. 

Figure 43 shows a vertical motion vector impulse where three vertically aligned pixels have a motion 
vector different from 0iose of the surrounding pixels. 

Figure 44 shows a diagonal motion vector impulse where three diagonally aligned pixels have a motion 

20 vector different from those of all the sunrounding pixels. 

Figure 45 diows a horizontal plus verttcal motion vector impuise, where five pix^ disposed in an 
upright cross have a motion vector (fifterent tirom those of aU ttw sunrounding pixels. 

Rgure 46 shows a two-diagonal motion vector impulse where five pixels arranged In a diagonal cross 
have a motion vector different from those of all the surrounding pixels. 

2s It is assumed that pixel motion vectors which tell Into any of the above six categories do not actually 
belong to a real picture, and are a direct result in of an incorrect motion vector seleclion. If such motion 
vectors were used during the interpolation (xocess. then they would be likely to cause d^ on the final 
output picture, and it is tfierefore preferable that such motion vectors be Identified and eliminated. Thte is 
done using an algorithm which will detect and flag all of the above moifion vector groupings. 

so The algorithm uses a twDi>ass process, with each pass being idendeaL ITie need for two passes will 
become apparent Figure 47, to which reference is made, shows an array of pixels, aD those marind wiSi a 
triangle having the same motian vector anodated therewith. The Uock of nine pixels in the centre has 
moSon vectors designated vector 1 to vector 8 associatod therewith, which motion vectors may or may not 
be the same. Vector 5 is the mctfon vector under tesL 

98 b) the first pass, vector Sis chedesd to detsmiinewheewr ft is the same as, or within a prsdetermined 
tolerance of: 
firsUy 

vector 1 or vector 3- or vector 7 or vector 9 
andsecondly 
40 vector 2 or vector 4 or vector 6 or vector 8 

This checks to see if vector 5 is the same as at toast one ofite horizontel or verttoal neighbours, and the 
same as at toast one of ite (fiagonal neighbours. If this to not the case, then a llag to set to Imficate that 
pixel 5 is bad. 

The first pass will flag as bad those moSon vectors relating to point singularities, horizontal motion 
45 vector impulses, vertical motton vector impulses, diagonal motion vector impulses and two diagonal motion 
vector impulses (Figures 41 to 44 and 46). but not the motion vectors corresponding to horizontal plus 
vertical motion vector impulses (Figure 48^ «or whfch pass 2 Is required. The second pass checks for 
exactly the same contfittons as to the ftst pass, but in thte ewe motton vectors which have akieady been 
flagged as bad are not included in the catoulaOon. Thus, retening to Figure 45. ^ the first pass only the 
so ceiitre mofion vector is fiagged as bad. but after the sectond pass an five of the motian v 
the upright cross are flagged as bad. 

Having ktentlfled the bad motton vectors, K is then necessary to repair them, this also b^ effected by 
the motkxi vector post processor 7 (Figure 1). Although vwtous methods such as interpolation or majority 
replacement can be used, it is has been found that in practtee simple replacemert gives good results. This 
55 is effected as foltows (aid it shouU be noted ttwt the 'equals' signs mean not only exactly equal to. but ako 
betog within a predetermined tolerance of): If vector 5 is flagged as bad then it Is replaced with: 
vector 4 ff (vector 4 equals vector ^ 
else with vector 2 if (vector 2 equals vector 8) 
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else with vector 1 if (vector 1 equals vector 8) 
elsft with vector 3 if (vector 3 ec^ials vector 7) 
else do nothing 

Referring again to Figure 1. the finally seiectBd motion vector for each pixel Is supplied by the motion 
s vector post processor 7 to the intapcMor 8. together with the progressive scan converted frames at 50 
frames per second from the progressiva scan converter 2. The MerpolatDr 8 is of relatively simple fonn 
using only two progressive scan converted frames, as indicated in Rgure 48u Ifeing the temporal position of 
the output frame relative to successive input frames, frame 1 and frame 2, and the motion vector for the 
pixel in the ou^Mit frame, the interpolator 8 detennines in lawwn manner which part of tlie lirst frame should 
10 be combined with which pat of the second frwie and with what weighting to produce the conrect output 
pixel vidue. In other words, the interpotetor 8 adaptlvely interpolates along the direction of movement in 
dependence on the motion vectors to procHice motion compensated progressh^ scan frames corresponding 
to 24 frames per second. Although the moflon vectors have been derived using only luminance values of 
the pixels, the same motion vectors are used for deriving the required output pixel chrominance values. An 
75 8x8 anray of pixels are used from each frame to produce the required output. Thus the interpolator 8 is a 
two-dimensional, vertical/horizontal, interpdaior and the coefDdents used for the interpolator 8 may be 
derived using tfie Remez exchange algorithm which can be found ftilly wqplained in TTwory and application 
of digital signal processing', Lawrence R Rabiner, eemard Gold. PrenUoe-Hall Inc., pages 196 to 140 and 
227. 

so Figure 48 Shows diagfammalically the inlerpolalion performed by the Interpolator 8 (Figure 1) for three 
different cases. The first case, shown on Die leK. is where there are no uncovered or covered surfaces, the 
second case, shown in ttie cemre, Is wtiere there is a covered surface, and the third case, shown on the 
right, is where there is an uncovered surface. In the case of a covered surl^, the intorpolafion uses only 
-frame 1, whilst in the case of an uncovered swface, the interpolafion uses only firame 2. 

2s Provision can be made in the interpolator 8 to defeujit to nonaction compensated inlerpolsiion, in which 
case the temporally nearest progressive scan converted frame Is used. 

Attention is drawn to ttie fact that this application Is one of a series of fourteen filed on the same day, 
and bearing our references E241-5 (con'esponding to UK 89096432), E241-6 (corresponding to UK 
8809644.0). E241-7 (conesponding to UK 8909645.7), E241-8 (conesponding to UK 8909646.5). E241-g 

30 (corresponding to UK 8909647.3). E241-10 (con^e^wnding to (iK 8009846.1), E241-11 (corresponding to UK 
8909649.9). E241-12 (conesponding to UK 89096S0.7). £241-13 (conesponding to UK 8908651.5). E241-14 
(corresponding to UK 8908652.3). E242-1 (correspond^ to UK 8808653.1), E242-2 (conesponding to UK 
8809654.9), E242^ (corresponding to UK 8809655.6) and E2424 (conesponding to UK 8809656.4). These 
applications all relate to sfmifBr sut^ect matter, and the disclosure in each is inco f poratad by ttite reference 

S6 Into each other. 



ClainM 

40 1. Amotion oompeMatod video Standards converter comprising: 

means (3. 4) for comparing blocks in a first field or frame of a video ^ai with a plurality of btocks in 

UOotdng HM or $rm» of the video signal tor deriving motion vectors representtng the mofion of the content 

of re^Mctive said btodts between said first field or frame and said toliowhg fieid orftame; 

means (27) to assign to each of said bioclts acQacent to a border region of each said field or frame, motion 
4S vectors selected from motion vectors which have been derived for those of said tiioclcs which are ac^acent 

to said Mode under consideration and are not in said border region; and 

an interpolator (8) controlled In dependence on said motion vector. 

Z A converter according to dalm 1 wlierein said mmns (p7) assigns to each of said blocks adjacent to 

a comor of said border region, motion vectors setected from the motion vectors v»Mch have been derived 
50 for the three said blocics wtilch are ad iace wt to said i^ock under consideration and are not in said border 

3. A converter accorcRng to dafcn 1 or claim 2 wherein said meane (27) assigns to each of said bioclcs 
adjacent to an edge of said border region, motion vectors selected from ttte mofion vectors which have 
Cteen derived for the five said bkx:te which are adiacent to said bloci( under consideration and are not in 

ss said bordw region. 

4. A converter acoordbig to daim 1, claim 2 or claim 3 wherein s^ means (27) additionally assigns a 
zero motion vector to said block under consideration. 

5. A oonvenar according to any one of the preceding elaime wherein said comparison is effected by 
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comparing the sums of the limiinance levsis of tiie pixols hi the fe^ 

B. A method of deriving motion vectors r^esenttng motion beltmBn successive fields or frames of a 
video signal, the method including the steps of: 

comparing blocks in a first field or frame of the video signal wWi a plurBiity of blocks in the following field or 
frame of the video signal for deriving said motion vectors representing the motion of the content of 
respective said blocks between saWfirst field or frame and said following fieW or frame: and 
assigning to each of said blocks (a: q) adjacent to a border region of each said field or frame, the motion 
vectors selected from the molton vectors wf^ have been derived for those of s^ btocks (d, b, c; r, f, s, t, 
g) whk:h are adfacent to saM btock (a. q) under consMerstion and are not in said bwder region. 

7. A mettiod aoooning to daim 6 wherah saM assigrvneiit compifses assigning to each of saM titoOta 
(a) adjacent to a comer of said border region, motion vectors selected from the motton vectors whteh have 
been derived for me three saki btocks (d. b. c) whtoh are adjacent to saM bfock under consktoraSon and 
are not in said border regton. 

8. A method according to daim 6 or claim 7 wherein saM assignment comprises assigning to each of 
said blocks (q) adjacent to an edge of s^ border regm, motton vectors selected from the motton vectors 
whteh have been derived for the five saW btocks (r. f, s. t. g) whfch are aitjacem to sakl btock {q) under 
conskleratkx) and are not in said border region. 

8. A method according to daim 6, ciatm 7 or daim 8 oompiising addittonaliy assigning a zero motion 
vector to said block <a, q) under conskferatkm. 

10. A method according to any one of daima 6 to 9 wherein said comparison is effected by comparing 
the sums d the kiminance tovete d the pixels to me respective saM btods to be con^iarad. 
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